A Two Step Viscous Damping Method to Solve a Class of Second Order Optimal Control Problems

This paper presents a method to solve the Hamilton-Jacobi-Bellman equation for a class of second order optimal control problems for which the cost is quadratic and the dynamics are affine in the input and in the state directly excited by that input. The solution is based on the concept of inverse optimality and is obtained in two steps. First a scalar problem is solved and then the solution of the second order problem is obtained from the solution to the scalar problem by the addition of a viscous damping term. The running cost that makes the control input optimal is explicitly determined in the second step of the method. A special feature of this method, as compared to other methods in the literature, is the fact that the solution is obtained directly for the control input without needing to assume or compute a value function first. Additionally, the value function can also be obtained after solving for the control input. Three important special cases are highlighted: massspring dynamics, Van der Pol oscillator and integrator backstepping. A Lyapunov function that proves stability of the controller is also constructed for these cases.